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APPARATUS AND METHOD FOR TRANSMITTING SYNCHRONIZED THE FIVE 

SENSES WITH A/V DATA 
Description 

Technical Field 

The present invention relates to an apparatus and 
method for synchronizing and transmitting five sensory data 
and an actual~f eeling multimedia data providing system and 
method; and;. more particularly, to a five sensory 
synchronizing and transmitting apparatus and method which 
forms packets by describing vibration, odor, and taste 
expressed in video/audio by using touch, odor and taste 
data descriptor, synchronizes touch/odor/taste packets with 
video/audio packets on a frame basis and transmitting the 
synchronized packets, and an actual-feeling multimedia data 
providing system and method that can provide an actual- 
feeling multimedia service by demultiplexing the packets 
transmitted from the five sensory data synchronizing and 
transmitting apparatus and transmitting video data, audio 
data, touch data, odor data and taste data to corresponding 
devices . 

Background Art 

Recent development in digital video/audio technology 
provides more realistic three-dimensional video and 
stereophonic sound and, further, an actual-feeling 
multimedia service applying all of the five senses of a 
human being stands in the spotlight. 

Korean Patent Laid-open Nos. 2001-0096868 (which 
relates to a vibration effect device) and 2001-0111600 
(which relates to a movie presenting system) disclose the 
actual-feeling multimedia service technology. 

The vibration effect device stores vibration signals 
expressed in video by using the number of frames of the 
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video or time code in a memory in advance and applies the 
stored vibration signals to a user whenever scenes of the 
video is out putted. 

The movie presenting system provides a vibration 
device that provides vibration signals to a user according 
to the intensity of audio sound outputted from speakers 
when a movie is shown in a theater and the like. 

The conventional technologies do not precisely 
describe the direction and rotation with respect to motion 
of a person or an object expressed in the video/audio and 
only gives the users vibration by using the vibration 
effect device for a predetermined video/audio play time or 
by using the vibration device according to the intensity of 
audio sound. 

However, since the conventional technologies do not 
precisely describe the direction and rotation with respect 
to motion of a person or an object expressed in the 
video/audio, there is a problem that the user enjoying the 
video/audio cannot enjoy the sense of vibration delicately 
and accurately. Also, since the conventional technologies 
do not describe odor and taste which are expressed in the 
video/audio, they fail to provide the users with a 
realistic actual-feeling multimedia service. 

Meanwhile, under development is technology for 
spraying chemical aromatics to the users enjoying the 
video/audio by using an odor device and releasing taste 
forming materials to users by using a -taste device whenever 
scenes (or circumstances) are changed. However, the odor 
device and the taste device cannot express the exact odor 
and taste presented in the video/audio and the spray and 
the chemical aromatics are sprayed and released by 
arbitrary manipulation of the users. Also, an actual- 
feeling multimedia data providing system, which is under 
development at present, the vibration, odor and taste are 
not synchronized with the video and sound presented in the 
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video/audio, and they are simply described in a level 
similar to each scene. 



Disclosure 



Technical Problem 



It is, therefore, an object of the present invention 
to provide a five sensory synchronizing and transmitting 
apparatus which forms packets by describing vibration, odor, 
and taste expressed in video/audio by using touch, odor and 
taste data descriptors , synchronizes touch/ odor /taste 
packets with video/audio packets on a frame basis and 
transmitting the synchronized packets, and a method thereof. 

It is another object of the present invention, there 
is provided an actual-feeling multimedia data providing 
system that can provide an actual-feeling multimedia 
service by demultiplexing the packets transmitted from the 
-five sensory data synchronizing and transmitting apparatus 
and transmitting video data, audio data, touch data, odor 
data and taste data to corresponding devices, and a method 
thereof. 

In accordance with one aspect of the present invention, 
there is provided an apparatus for synchronizing and 
transmitting five sensory data, which includes: a 
video/audio data generating unit for generating video/audio 
data by receiving multimedia data from an external device; 

a touch data describing unit for describing vibration 
expressed in the multimedia data received from the external 
device based on a predefined touch data descriptor; an odor 
data describing unit for describing an odor expressed in 
the multimedia data transmitted from the external device 
based on a predefined odor data descriptor; a taste data 
describing unit for describing a taste expressed in the 
multimedia data transmitted from the external device based 
on a predefined taste data descriptor; a video/audio packet 
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forming unit for forming video/audio packets out of the 
video/audio data generated in the video/audio generating 
unit; a touch/odor /taste packet forming unit for forming a 
touch packet, an odor packet, and a taste packet out of the 
touch, odor and taste data which are described in the touch 
data describing unit, the odor data describing unit, and 
the taste data describing unit, respectively; a 
multiplexing unit for multiplexing the video/audio packets 
generated in the video/audio packet generating unit with 
the touch packet, the odor packet and the taste packet 
formed in the touch/odor/taste packet forming unit to 
thereby synchronize the video/audio packets with the 
touch/odor /taste packets; and a transmitting unit for 
transmitting a multiplexed packet multiplexed in the 
multiplexing unit. 

In accordance with one aspect of the present invention, 
there is provided a method for synchronizing and 
transmitting five sensory data, which includes the steps 
of: a) generating video/audio data by receiving multimedia 
data from an external device; b) describing vibration, 

an odor and a taste expressed in the multimedia data 
transmitted from the external device to generate touch data, 
odor data and taste data based on predefined touch, odor 
and taste data descriptors, respectively; c) forming 
video/audio packets out of the video/audio data; and 
forming a touch packet, an odor packet and a taste packet 
out of the touch data, the odor data and the taste data, 
respectively; d) performing synchronization by multiplexing 
the video/audio packets, the touch packet, the odor packet 
and the taste packet; and e) transmitting a multiplexed 
packet to a receiving part. 

In accordance with one aspect of the present invention, 
there is provided a system for providing actual-feeling 
multimedia data, which includes: a video/audio data 
generating unit for generating video/audio data by 
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receiving multimedia data from an external device; a touch 
data describing unit for describing vibration expressed in 
the multimedia data transmitted from the external device 
based on a predefined touch data descriptor; an odor data 
describing unit for describing an odor expressed in the 
multimedia data received from the external device based on 
a predefined odor data descriptor; a taste data describing 
unit for describing a taste expressed in the multimedia 
data received from the external device based on a 
predefined taste data descriptor; a video/audio packet 
forming unit for forming video/audio packets out of the 
video/audio data generated in the video/audio generating 
unit; a touch/odor /taste packet forming unit for forming a 
touch packet, an odor packet, and a taste packet out of the 
touch, odor and taste data described in the touch data 
describing unit, the odor data describing unit, and the 
taste data describing unit, respectively; a multiplexing 
unit for multiplexing the video/audio packets generated in 
the video/audio packet generating unit and the touch packet, 
the odor packet and the taste packet formed in the 
touch/odor/taste packet forming unit to thereby synchronize 
the video/audio packets with the touch/odor/taste packets; 
a transmitting unit for transmitting a multiplexed packet 
obtained in the multiplexing unit; a receiving unit for 
receiving the multiplexed packet; a demultiplexing unit for 
demultiplexing the multiplexed packet received by the 
receiving unit into the video data, the audio data, the 
touch data, the odor data and the taste data; a video 
device for decoding and outputting the video data 
demultiplexed by the demultiplexing unit; an audio device 
for decoding and outputting the audio data demultiplexed by 
the demultiplexing unit; a vibration device for providing 
vibration to a user by interpreting the touch data 
demultiplexed by the demultiplexing unit; an odor device 
for spraying chemical aromatics to a user by interpreting 
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the odor data demultiplexed by the demultiplexing unit; and 
a taste device for releasing a taste forming material to a 
user by interpreting the taste data demultiplexed by the 
demultiplexing unit. 

In accordance with one aspect of the present invention, 
there is provided a method for providing actual-feeling 
multimedia data in an actual-feeling multimedia data 
providing system, which includes the steps of: a) 
generating video/audio data by receiving multimedia data 
from an external device; b) describing vibration, an odor 
and a taste expressed in the multimedia data transmitted 
from the external device to thereby generate touch data, 
odor data and taste data based on predefined touch, odor 
and taste data descriptors, respectively; c) forming 
video/audio packets out of the video/audio data; and 
forming a touch packet, an odor packet and a taste packet 
out of the touch data, the odor data and the taste data, 
respectively; d) performing synchronization by multiplexing 
the video/audio packets with the touch packet, the odor 
packet and the taste packet; e) transmitting a multiplexed 
packet to a receiving part; f) receiving the multiplexed 
packet and demultiplexing the multiplexed packet received 
by the receiving unit into the video data, the audio data, 
the touch data, the odor data and the taste data; g) 
decoding and outputting the demultiplexed video data and 
the demultiplexed audio data; h) providing a user with 
vibration by interpreting the demultiplexed touch data; i) 
spraying chemical aromatics to the user by interpreting the 
demultiplexed odor data; and j) a taste device for 
releasing taste forming materials to a user by interpreting 
the demultiplexed taste data. 

Description of Drawings 

The above and other objects and features of the 
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present invention will become apparent from the following 
description of the preferred embodiments given in 
conjunction with the accompanying drawings, in which: 

Fig. 1 is a block diagram illustrating a five sensory 
data synchronizing and transmitting apparatus and a real- 
sense multimedia data providing system using the same in 
accordance with an embodiment of the present invention; 

Fig. 2A describes a touch data descriptor in 
accordance with an embodiment of the present inventions- 
Fig. 2B is a diagram showing a header of a touch 
packet in accordance with an embodiment of the present 
invention; 

Fig. 3A is a diagram describing an odor data 
descriptor in accordance with an embodiment of the present 
invention; 

Fig. 3B is a diagram showing a header of an odor 
packet in accordance with an embodiment of the present 
invention; 

Fig. 4A is a diagram describing a taste data 
descriptor in accordance with an embodiment of the present 
invention; 

Fig. 4B is a diagram showing a header of a taste 
packet in accordance with an embodiment of the present 
invention; and 

Fig. 5 is a flowchart describing a five sensory data 
synchronizing and transmitting method and a real-sense 
multimedia data providing system using the same in 
accordance with an embodiment of the present. 

Best Mode for the Invention 

Other objects and aspects of the invention will become 
apparent from the following description of the embodiments 
with reference to the accompanying drawings, which is set 
forth hereinafter. 
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Fig. 1 is a block diagram illustrating a five sensory 
data synchronizing and transmitting apparatus and a real- 
sense multimedia data providing system using the same in 
accordance with an embodiment of the present invention. 

As illustrated in Fig. 1, in the real-sense multimedia 
data providing system of the present invention, the five 
sensory data synchronizing and transmitting apparatus, 
which is a transmitting part 100, comprises a video/audio 
data generating module 10, a video/audio packet forming 
module 11, a touch data describing module 12, an odor data 
describing module 13, a taste data describing module 14, a 
touch/odor /taste packet forming module 15, a multiplexing 
module 16, and a transmitting module 17. The video/audio 
data generating module 10 receives multimedia data provided 
from an external device of a contents provider and generate 
video/audio data having a compressed stream type by using 
video encoding method, such as Moving Picture Experts Group 
2 (MPEG-2) compressed encoding method. The video/audio 
packet forming module 11 forms the stream type of 
video/audio data generated in the video/audio data 
generating module 10 into packets suitable for a 
transmission method. The touch data describing module 12 
describes vibration expressed in the multimedia data 
provided form the external device of the content provider 
by using a pre-defined touch data descriptor. The odor 
data describing module 13 describes odor expressed in the 
multimedia data provided form the external device of the 
content provider by using a pre-defined odor data 
descriptor. The taste data describing module 14 describes 
taste expressed in the multimedia data provided form the 
external device of the content provider by using a pre- 
defined taste data descriptor. The touch/odor/taste packet 
forming module 15 forms the touch/odor/taste data described 
in the touch data describing module 12, odor data 
describing module 13, and taste data describing module 14 
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into packets suitable for a transmission method. The 
multiplexing module 16 multiplexes the video/audio packets 
formed in the video/audio packet forming module 11 and the 
touch/odor/taste packets formed in the touch/odor/taste 
packet forming module 15 based on each frame. The 
transmitting module 17 transmits the packets multiplexed in 
the multiplexing module 16 to a receiving part 200. 

Meanwhile, the receiving part 200 comprises a 
receiving module 20, a demultiplexing module 21, a 
video/audio decoding module 22, a video device 23, an audio 
device 24, a vibration device 25, an odor device 26, and a 
taste device 27. The receiving module 20 receives the 
stream-type packets transmitted from the transmitting part 
100. The demultiplexing module 2^1 depacketizes the packets 
received in the receiving module 20, demultiplexes the 
resultant into the video data, audio data, touch data, 
taste data and odor data, and transmits the data to 
corresponding processing devices. The video/audio decoding 
module 22 decodes video data and audio data demultiplexed 
in the demultiplexing module 21. The video device 23 
outputs the video data decoded in the video/audio decoding 
module 22 onto a screen. The audio device 24 outputs the 
video data decoded in the video/audio decoding module 22 
onto a screen. The vibration device 25 receives touch data 
demultiplexed in the demultiplexing module 21 and gives 
vibration to the user to feel movement and rotation. The 
odor device 2 6 receives odor data demultiplexed in the 
demultiplexing module 21, spraying chemical aroma to the 
user to feel the odor. The taste device 27 receives taste 
data demultiplexed in the demultiplexing module 21, 
releasing chemical taste forming materials to the user to 
feel the taste. 

Herein, the real-sense multimedia data providing 
system of the present invention includes the transmitting 
part 100 and the receiving part 200. 
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Hereinafter, structures and operations of the 
structural elements will be described in detail . 

The video/audio packet forming module 11 forms 
video/audio packets, each of which is formed of a header 
and payloads, to be suitable for transmitting the 
video/audio data having a compressed stream type generated 
in the video/audio data generating module 10 through a 
communication network. Herein, the header contains a 
destination address, data for checking continuity when data 
are lost, data for controlling time synchronization such as 
time stamp and the payloads contains the video/audio data 
having the compressed stream type. 

The touch data describing module 12 describes 
vibration expressed in the multimedia data provided from 
the external device of the content provider by using 
descriptors describing where touch data are described, 
whether right /left movement is described, whether up/down 
movement is described, whether back/forth movement is 
described, movement distance, movement velocity, movement 
acceleration, whether right/left rotation is described, 
right/left rotation angle, right/left rotation speed, and 
right/left rotation acceleration. 

The odor data describing module 13 describes odor 
expressed in the multimedia data provided from the external 
device of the contents provider by using descriptors for 
whether odor data are described, kind of odor, and 
intensity of odor. 

The taste data describing module 14 describes taste 
expressed in the multimedia data provided from the external 
device of the contents provider by using descriptors for 
whether taste data are described, kind of taste, and taste 
intensity. 

For example, when a producer related to a real-sense 
movie service provided to the receiving part 200 sees a 
pre-produced movie, the producer describes vibration, odor 
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and taste of a current scene of the movie in the form of 
touch/odor/taste data by using touch data descriptors, odor 
descriptors, and taste descriptors to be suitable for the 
scene, synchronizes the touch/odor/taste data with the 
video data and audio data, and transmits the synchronized 
data to the receiving part 200. Also, it is possible that 
not all touch/odor/taste data can be described for one 
scene or that the touch/odor/taste data are combined and 
then described. 

The touch/odor/taste packet forming module 15 forms 
the stream-type touch/odor/taste data, which are described 
in the touch data describing module 12, odor data 
describing module 13, and taste data describing module 14 
by using corresponding touch/odor /taste descriptors, into 
packets including a header which are suitable forms to be 
transmitted to the receiving part 200 through the network. 
Herein, the header includes descriptor information that 
describes the touch/odor /taste data. The packets formed in 
the touch/odor/taste packet forming module 15 includes the 
touch/odor /taste data sequentially. 

The multiplexing module 16 synchronizes the 
video/audio packet and the touch/odor/taste packet formed 
in the video/audio packet forming module 11 and the 
touch/odor/taste packet forming module 15. The 
multiplexing module 16 performs multiplexing by adding all 
the video/audio packets into frames that form the 
multimedia data and adding the touch/odor/taste packets 
into the last packet. That is, one frame is formed of a 
plurality of video/audio packets. Among the packets of 
each frame, the touch/odor /taste packet is added to the 
last packet. In short, the touch data, the odor data and 
the taste data are added to the last packet of each frame 
sequentially . 

The demultiplexing module 21 of the receiving part 200 
depacketizes the stream-type packet received in the 
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receiving module 20, demultiplexes into video/audio data 
formed of a payload and a header deprived of network- 
related header information, e.g., address of the 
transmitting part 100, and into touch/odor /taste data 
formed of a header, and transmits the data to corresponding 
processing devices. Herein, the demultiplexing module 21 
examines the headers of the received packets and confirms 
whether the data of packet is video data, audio data, touch 
data, odor data, and taste data- In other words, video 
data and audio data that form one frame are all transmitted 
to corresponding processing devices and then the touch data, 
the odor data, and the taste data are transmitted to 
corresponding processing devices sequentially to thereby 
synchronize five sensory data, i.e., video data, audio data, 
touch data, odor data, and taste data and make a user feel 
vibration, odor and taste expressed in the circumstance of 
each scene of the multimedia data along with video and 
sound. 

The vibration device 25 is embodied as a vibration 
chair that can be moved right and left, up and down, and 
back and forth and/or rotated. The vibration device 25 
reads the touch data which is demultiplexed (or separated) 
in the demultiplexing module 21 and makes a movement or 
rotation in the right and left, up and down and back and 
forth. Herein, the starting time and duration of the 
movement or rotation of the vibration device 25 is 
determined by being synchronized with the video and sound 
outputted from the video device 23 and the audio device 24. 
That is, as the transmitting part 100 transmits the touch 
data for video and sound, the vibration device 25 reads the 
transmitted touch data and makes a movement in the 
requested direction or makes a rotation. Then, if the 
transmitting part 100 transmits another touch data for 
another video and sound, the vibration device 25 reads the 
new touch data transmitted thereto, stops previous movement 
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and makes a movement in a different direction or makes a 
rotation. 

The odor device 26 is embodied as an aroma spray which 
is provided with a plurality of chemical aromatics and it 
can control the intensity of the odor. It analyzes the 
odor data demultiplexed, or separated, in the 
demultiplexing module 21 and sprays chemical aromatics 
having a corresponding intensity. Herein, the starting 
time and duration of the spraying of a specific chemical 
aromatic in the odor device 2 6 are determined after 
synchronized with video and sound outputted from the video 
device 23 and the audio device 24. In addition, the odor 
device 2 6 can spray one kind of odor by mixing a plurality 
of chemical aromatics or spray a plurality of prepared 
aromatics simultaneously to spray diverse aromatics 
corresponding to the odor data described in the 
transmitting part 100. 

The taste device 27 is embodied in such a method that 
a plurality of chemical taste forming materials are 
prepared and a chemical taste forming material of the 
corresponding taste is released into the mouth of a user 
through a straw. The taste device 27 analyzes the taste 
data demultiplexed, or separated, in the demultiplexing 
module 21 and releases a chemical taste forming material of 
the corresponding taste. Herein, the starting time and 
duration of the release of a specific chemical taste 
forming material in the taste device 27 are determined 
after synchronized with video and sound outputted from the 
video device 23 and the audio device 24. 

Fig. 2A describes a touch data descriptor in 
accordance with an embodiment of the present invention, and 
Fig. 2B is a diagram showing a header of a touch packet in 
accordance with an embodiment of the present invention. 

A touch object flag (TouchObj ectFlag) indicates 
whether or not there is a touch data description. For 
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example, when the touch object flag (TouchObjectFlag) is 1, 
it means that the touch data are described and, accordingly, 
the touch data are transmitted from the demultiplexing 
module 21 of the receiving part 200 to the vibration device 
25, thereby activating the vibration device 25. 

A length field indicates the size of the touch data 
packet and the size is 64 bits. 

An X_MoveFlag indicates whether or not there is a 
description on the right/left movement in the touch data. 
For example, when the X_MoveFlag is 1, the vibration device 
25 moves in the right/left. 

An Y^MoveFlag indicates whether or not there is a 
description on the up/down movement in the touch data. For 
example, when the Y__MoveFlag is 1, the vibration device 25 
moves up and down. 

A Z_MoveFlag indicates whether or not there is a 
description on the back/forth movement in the touch data. 
For example, when the Z_MoveFlag is 1, the vibration device 
25 moves back and forth. 

Herein, only any one move flag among the X__MoveFlag, 
Y_MoveFlag and Z_MoveFlag is activated for a predetermined 
time. Thus, the vibration device 25 moves only in one 
direction among right/left, up/down and back/forth. 

A MoveDistance indicates a distance of movement in any 
one direction among the right/left, up/down and back/forth 
in the touch data. In other words ^ as any one move flag 
among the X_MoveFlag, Y_MoveFlag and Z__MoveFlag is 
activated, the MoveDistance indicates a movement distance 
in a direction corresponding to the MoveFlag. For example, 
if X_MoveFlag is 1 and the MoveDistance is 10cm, the 
vibration device moves in the right and left range of 10cm. 

A MoveSpeed indicates a speed of movement in one 
direction among right/left, up/down and back/forth in the 
touch data. For example, if the X_MoveFlag is 1 and the 
MoveDistance is 10cm and the MoveSpeed is 5cm/second,- the 
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vibration device 25 moves in the right and left range of 
10cm for 2 seconds. 

The MoveAcceleration indicates an acceleration of 
movement in any one direction among the right /left, up/down 
and back/forth. For example, if the X_MoveFlag is 1 and 
the MoveDistance is 10cm and the MoveSpeed is 5cm/second 
and the MoveAcceleration is 5cm/second^, the vibration 
device 25 moves in the right and left range of 10cm for 2 
seconds and the movement is increased gradually at an 
acceleration of 5cm/second^. 

A RotationFlag indicates whether or not there is 
right /left rotation description. For example, if the 
RotationFlag is 1, the vibration device 25 is rotated 
right/left . 

A RotationAngle indicates a right/left rotation angle 
in the touch data. 

A RotationSpeed indicates a right/left rotation speed 
in the touch data. 

A RotationAcceleration indicates right/left rotation 
acceleration in the touch data. 

Fig. 3A is a diagram describing an odor data 
descriptor in accordance with an embodiment of the present 
invention; and Fig. 3B is a diagram showing a header of an 
odor packet in accordance with an embodiment of the present 
invention. 

A SmellObjectFlag indicates whether or not there is an 
odor data description. For example, if the SmellObjectFlag 
is 1, it means that the odor data are described and, 
accordingly, the odor data are transmitted from the 
demultiplexing module 21 of the receiving part 200 to the 
odor device 2 6 to thereby activate the odor device 26. 

A length field indicates the size of an odor data 
packet and the size is 32 bits. 

A ^Type^ means the kind of odor in the odor data. For 
example, the odor of an aroma is pre-established as ^100'' 
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and if the SmellObjectFlag is 1 and the type is 100, the 
odor device 26 sprays a chemical aromatic having the odor 
of the aroma. 

A ^Level' indicates the intensity of the odor in the 
odor data. For example, if the SmellObjectFlag is 1 and 
the type is 100 and the level is 31, the odor device 26 
sprays a chemical aromatic having the odor of the aroma at 
the predetermined level of 31. Herein, the higher the 
level is, the stronger the intensity of the odor is. 

Fig. 4A is a diagram describing a taste data 
descriptor in accordance with an embodiment of the present 
invention; and Fig. 4B is a diagram showing a header of a 
taste packet in accordance with an embodiment of the 
present invention . 

A TasteObjectFlag indicates whether or not there is a 
taste data description. For example, if the 

TasteObjectFlag is 1, it means that the taste data are 
described and, accordingly, the taste data are transmitted 
from the demultiplexing module 21 of the receiving part 200 
to the taste device 27 to thereby activate the taste device 
27. 

A 'Length' field indicates the size of a taste data 
packet and the size is 32 bits. 

A 'Type' indicates the kind of taste in the taste data. 
For example, if a hot taste is pre-established as '7' and 
if the TasteObjectFlag is 1 and the type is 7, the taste 
device 27 releases a chemical taste forming material that 
tastes hot. 

A ^Level' indicates the intensity of taste in the 
taste data. For example, if the TasteObjectFlag is 1 and 
the type is 7 and the Level is 31, the taste device 27 
releases a chemical taste forming material that tastes hot 
with an intensity of the pre-established 31. 

Fig. 5 is a flowchart describing a five sensory data 
synchronizing and transmitting method and a real-sense 
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multimedia data providing system using the same in 
accordance with an embodiment of the present. 

First, at step 500, multimedia data are inputted from 
an external device, e.g., a contents provider - 

At step 501, video/audio data having a compressed 
stream type are generated. In other words, when multimedia 
data are inputted from an external device, e.g., a contents* 
provider, compressed stream-type video/audio data are 
generated by using an image encoding method, such as Moving 
Picture Experts Group 2 (MPEG-2) compressed encoding method. 

Subsequently, at step 503, the stream-type video/audio 
data, which are generated in the above, are formed into 
video/audio packets. That is, the stream-type video/audio 
data are formed into video/audio packets which are formed 
of a header including destination address information and a 
payload including substantial video/audio data, which are 
proper forms to transmit the stream-type video/audio data 
to the receiving part 200 through a network. 

Meanwhile, at step 502, the vibration/odor/taste 
expressed in the inputted multimedia data are described by 
using touch/odor/taste descriptors. That is,* vibration 
expressed in the multimedia data provided from the external 
device, e.g., a contents provider, is described by using a 
predefined touch descriptor, and the odor expressed in the 
multimedia data provided from the external device, e.g., a 
contents provider, is described by using a predefined odor 
descriptor, while the taste expressed in the multimedia 
data provided from the external device, e.g., a contents 
provider, is described by using a predefined taste 
descriptor. 

Subsequently, at step 504, the touch/odor/taste data 
are formed into touch/odor/taste packets. That is, 

touch/odor/taste packets having a header including 
touch/odor/taste data descriptor information sequentially 
are formed so that the above described touch data, odor 



17 



wo 2005/048541 



PCT/KR2003/002917 



data and taste data can be transmitted to the receiving 
part 200 through the network properly. 

Subsequently, at step 505, the audio/video packet and 
the touch/odor/taste packets are multiplexed on a frame 
bass. Herein, the multiplexing module 16 synchronizes the 
audio/video packets and the touch/odor/taste packets which 
are restructured in the audio/video packet forming module 
11 and the touch/odor /taste forming module 15, respectively - 
That is, the multiplexing module 16 sequentially performs 
the multiplexing by adding a plurality of audio/video 
packets to frames that forms the multimedia data and, 
lastly, adding the touch/odor /taste packets in order. 

At step 506, the multiplexed packets are transmitted 
to the receiving part 200. At step 507, the packets are 
received and demultiplexed into video/audio data and 
touch/odor/taste data in the receiving part 200. That is, 
the demultiplexing module 21 of the receiving part 200 
depacketizes the stream-type packets received in the 
receiving module 20 and finds out whether the packets are 
of video data, audio data, touch data, odor data and taste 
data by checking the headers of the received packets. 

At step 508, the demultiplexed video/audio data are 
decoded in the receiving part 200. 

Subsequently, at step 509, video data decoded in the 
receiving part 200 are transmitted to the video device 23. 

At step 510, audio data decoded in the receiving part 
200 are transmitted to the audio device 24, 

At step 511, touch data demultiplexed in the receiving 
part 200 in the step 507 are transmitted to the vibration 
device 25. 

At step 512, odor data demultiplexed in the receiving 
part 200 in the step 507 are transmitted to the odor device 
26. 

At step 513, taste data demultiplexed in the receiving 
part 200 in the step 507 are transmitted to the taste 
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device 27. 

Accordingly, at step 514, the video device 23 outputs 
the video data on a screen and, at step 515, the audio 
device 24 outputs the audio data on a speaker. At step 516, 
the vibration device 25 analyzes the touch data and gives 
vibration to the user to feel the sense of touch. At step 
517, the odor device 26 analyzes the odor data and sprays a 
chemical aromatic so that the user can feel the odor. At 
step 518, the taste device 270 analyzes the taste data and 
releases a chemical taste forming material so that the user 
can feel the taste - 

The method of the present invention, which is 
described above, can be embodied as a program and stored in 
a computer-readable recording mediiam, e.g., CD-ROM, RAM, 
ROM, floppy disks, hard disks, magnetooptical disks and the 
like. As the process can be easily implemented by those of 
ordinary skill in the art, further description on it will 
not be provided herein. 

Since the present invention describes vibration, odor, 
and taste expressed in multimedia data by using 
touch/odor/taste data descriptors and transmits them to 
corresponding devices on the user's part that receives the 
multimedia service, the user can receive more realistic 
real-sense multimedia service as well as sensing the five 
senses expressed in the multimedia data. 

Also, the present invention can provide the user with 
vibration, odor and taste that conform to each scene of the 
multimedia data with the vibration device, odor device and 
taste device by transmitting the synchronized video data, 
audio data, touch data, odor data and taste data based on 
each frame of the multimedia data. Therefore, the 
technology of the present invention can make the user feel 
the five senses expressed in each scene of the multimedia 
data precisely. 

While the present invention has been described with 
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respect to certain preferred embodiments, it will be 
apparent to those skilled in the art that various changes 
and modifications may be made without departing from the 
scope of the invention as defined in the following claims. 
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What is claimed is: 

1. An apparatus for synchronizing and transmitting 
five sensory data, comprising: 

a video/audio data generating means for generating 
video/audio data by receiving multimedia data from an 
external device; 

a touch data describing means for describing 
vibration expressed in the multimedia data received from 
the external device based on a predefined touch data 
descriptor; 

an odor data describing means for describing an odor 
expressed in the multimedia data transmitted from the 
external device based on a predefined odor data descriptor; 

a taste data describing means for describing a taste 
expressed in the multimedia data transmitted from the 
external device based on a predefined taste data 
descriptor; 

a video/audio packet forming means for forming 
video/audio packets out of the video/audio data generated 
in the video/audio generating means; 

a touch/odor/taste packet forming means for forming a 
touch packet, an odor packet, and a taste packet out of the 
touch, odor and taste data which are described in the touch 
data describing means, the odor data describing means, and 
the taste data describing means, respectively; 

a multiplexing means for multiplexing the video/audio 
packets generated in the video/audio packet generating 
means with the touch packet, the odor packet and the taste 
packet formed in the touch/odor/taste packet forming means 
to thereby synchronize the video/audio packets with the 
touch/odor/taste packets; and 

a transmitting means for transmitting a multiplexed 
packet multiplexed in the multiplexing means. 

2- The apparatus as recited in claim 1, wherein the 
touch data describing means describes vibration expressed 
in the multimedia data transmitted from the external device 
based on a descriptor describing whether touch data are 
described; a descriptor describing whether right/left 
movement is described; a descriptor describing whether 
up/down movement is described; a descriptor describing 
whether back/forth movement is described; a descriptor 
describing a distance of movement; a descriptor describing 
a speed of movement; a descriptor describing an 
acceleration of movement; a descriptor describing whether 
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right/left rotation is described; a descriptor describing 
an angle of right/left rotation; a descriptor describing a 
speed of right/left rotation; and a descriptor describing 
an acceleration of right/left rotation. 

3. The apparatus as recited in claim 2, wherein the 
odor data describing means describes an odor expressed in 
the multimedia data transmitted from the external device 
based on a descriptor describing whether the odor data are 
described; a descriptor describing a kind of the odor; and 
a descriptor describing an intensity of the odor. 

4. The apparatus as recited in claim 3, wherein the 
taste data describing means describes a taste expressed in 
the multimedia data transmitted from the external device 
based on a descriptor describing whether the taste data are 
described; a descriptor describing a kind of the taste; and 
a descriptor describing an intensity of the taste. 

5. The apparatus as recited in claim 1, wherein the 
touch/odor/taste packet forming means forms a touch packet 
including information on whether the touch data are 
described^ information on a packet length, and information 
on the touch data descriptors described in the touch data 
describing means; an odor packet including information on 
whether odor data are described,r information on an odor 
packet length, and information on the odor data descriptors 
described in the odor data describing means; and a taste 
packet including information on whether taste data are 
described, information on a taste packet length, and 
information on the taste data descriptors described in the 
taste data describing means. 

6. The apparatus as recited in claim 1, wherein the 
multiplexing means adds the touch/odor/taste packets formed 
in the touch/odor /taste packet forming means to the end of 
a plurality of video/audio packets generated in the 
video/audio generating means on a basis of the multimedia 
data frame to thereby multiplex and synchronize the 
video/audio packets with the touch/odor/taste packets. 

7- A method for synchronizing and transmitting five 
sensory data, comprising the steps of: 

a) generating video/audio data by receiving 
multimedia data from an external device; 

b) describing vibration^ an odor and a taste 
expressed in the multimedia data transmitted from the 
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external device to generate touch data, odor data and taste 
data based on predefined touch, odor and taste data 
descriptors, respectively; 

c) forming video/audio packets out of the video/audio 
data; and forming a touch packet, an odor packet and a 
taste packet out of the touch data, the odor data and the 
taste data, respectively; 

d) performing synchronization by multiplexing the 
video/audio packets, the touch packet, the odor packet and 
the taste packet; and 

e) transmitting a multiplexed packet to a receiving 

part. 

8. The method as recited in claim 7, wherein in the 
step b) , the vibration expressed in the multimedia data 
transmitted from the external device is described based on 
a descriptor describing whether touch data are described; a 
descriptor describing whether right/left movement is 
described; a descriptor describing whether up/down movement 
is described; a descriptor describing whether back/forth 
movement is described; a descriptor describing a distance 
of movement; a descriptor describing a speed of movement; a 
descriptor describing an acceleration of movement; a 
descriptor describing whether right/left rotation is 
described; a descriptor describing an angle of right/left 
rotation; a descriptor describing a speed of right/left 
rotation; a descriptor describing an acceleration of 
right/left rotation; 

the odor expressed in the multimedia data received 
from the external device is described based on a descriptor 
describing whether the odor data are described; a 
descriptor describing a kind of the odor; and a descriptor 
describing an intensity of the odor; and, 

the taste expressed in the multimedia data received 
from the external device is described based on a descriptor 
describing whether the taste data are described; a 
descriptor describing a kind of the taste; and a descriptor 
describing an intensity of the taste. 

9- The method as recited in claim 7, wherein in the 
step d) , the touch packet, the odor packet and the taste 
packet are added to the end of a plurality of video/audio 
packets on a basis of a multimedia data frame to thereby 
multiplex and synchronize the video/audio packets with the 
touch packet, the odor packet, and the taste packet, 

10- A system for providing actual-feeling multimedia 
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data, comprising: 

a video/audio data generating means for generating 
video/audio data by receiving multimedia data from an 
external device; 

a touch data describing means for describing 
vibration expressed in the multimedia data transmitted from 
the external device based on a predefined touch data 
descriptor; 

an odor data describing means for describing an odor 
expressed in the multimedia data received from the external 
device based on a predefined odor data descriptor; 

a taste data describing means for describing a taste 
expressed in the multimedia data received from the external 
device based on a predefined taste data descriptor; 

a video/audio packet forming means for forming 
video/audio packets out of the video/audio data generated 
in the video/audio generating means; 

a touch/odor/taste packet forming means for forming a 
touch packet, an odor packet, and a taste packet out of the 
touch, odor and taste data described in the touch data 
describing means, the odor data describing means, and the 
taste data describing means, respectively; 

a multiplexing means for multiplexing the video/audio 
packets generated in the video/audio packet generating 
means and the touch packet, the odor packet and the taste 
packet formed in the touch/odor/taste packet forming means 
to thereby synchronize the video/audio packets with the 
touch/odor/taste packets; 

a transmitting means for transmitting a multiplexed 
packet obtained in the multiplexing means; 

a receiving means for receiving the multiplexed 
packet; 

a demultiplexing means for demultiplexing the 
multiplexed packet received by the receiving means into the 
video data, the audio data, the touch data, the odor data 
and the taste data; 

a video device for decoding and outputting the video 
data demultiplexed by the demultiplexing means; 

an audio device for decoding and outputting the audio 
data demultiplexed by the demultiplexing means; 

a vibration device for providing vibration to a user 
by interpreting the touch data demultiplexed by the 
demultiplexing means; 

an odor device for spraying chemical aromatics to a 
user by interpreting the odor data demultiplexed by the 
demultiplexing means; and 

a taste device for releasing a taste forming material 
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to a user by interpreting the taste data demultiplexed by 
the demultiplexing means . 

11. The system as recited in claim 10, wherein the 
demultiplexing means deletes network-related information 
from the received packet in form of a compressed stream by 
depacketizing, divides the depacketized packet into the 
video data, the audio data, the touch data, the odor data 
and the taste data on a basis of a multimedia data frame, 
and transmits the video data, the audio data, the touch 
data, the odor data and the taste data to corresponding 
devices based on header information, 

12. The system as recited in claim 10, wherein the 
vibration device moves to right and left, back and forth, 
and up and down or rotates by interpreting the touch data, 
which are demultiplexed in the demultiplexing means, based 
on a predefined touch data descriptor; and a starting time 
and a duration time of movement or rotation operation are 
synchronized with a moving picture and a sound outputted 
from the video device and the audio device, respect ively. 

13. The system as recited in claim 12, wherein the 
odor device sprays the chemical aromatics by interpreting 
the odor data, which are demultiplexed in the 
demultiplexing means, based on a predetermined odor data 
descriptor; and a starting time and a duration time of 
spraying operation are synchronized with a moving picture 
and a sound outputted from the video device and the audio 
device , respectively . 

14- The system as recited in claim 13, wherein the 
taste device releases taste forming materials by 
interpreting the taste data, which are demultiplexed in the 
demultiplexing means, based on a predetermined taste data 
descriptor; and a starting time and a duration time of 
releasing operation are synchronized with a moving picture 
and a sound outputted from the video device and the audio 
device, respectively. 

15. A method for providing actual-feeling multimedia 
data in an actual-feeling multimedia data providing system, 
comprising the steps of: 

a) generating video/audio data by receiving 
multimedia data from an external device; 

b) describing vibration, an odor and a taste 
expressed in the multimedia data transmitted from the 
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external device to thereby generate touch data, odor data 
and taste data based on predefined touch, odor and taste 
data descriptors, respectively; 

c) forming video/audio packets out of the video/audio 
data; and f canning a touch packet, an odor packet and a 
taste packet out of the touch data, the odor data and the 
taste data, respectively; 

d) performing synchronization by multiplexing the 
video/audio packets with the touch packet, the odor packet 
and the taste packet; 

e) transmitting a multiplexed packet to a receiving 

part; 

f) receiving the multiplexed packet and 
demultiplexing the multiplexed packet received by the 
receiving means into the video data, the audio data, the 
touch data, the odor data and the taste data; 

g) decoding and outputting the demultiplexed video 
data and the demultiplexed audio data; 

h) providing a user with vibration by interpreting 
the demultiplexed touch data; 

i) spraying chemical aromatics to the user by 
interpreting the demultiplexed odor data; and 

j) a taste device for releasing taste forming 
materials to a user by interpreting the demultiplexed taste 
data. 
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FIG. 2A 



TouchObjectFlagdbit): WHETHER TOUCH DATA ARE DESCRIBED 
Length(64bits): PACKET SIZE OF TOUCH DATA 
X_MoveFlag(1bit): WHETHER RIGHT/LEFT MOVEMENT IS DESCRIBED 
Y_MoveFlag(1bit): WHETHER UP/DOWN MOVEMENT IS DESCRIBED 
Z_MoveFlag(1bit): WHETHER BACK/FORTH MOVEMENT IS DESCRIBED 
MoveDistance(5): DISTANCE OF MOVEMENT 
MoveSpeed(5): SPEED OF MOVEMENT 
MoveAcceleration(5): ACCELERATION OF MOVEMENT 
RotatlonFlagdbIt): WHETHER RIGHT/LEFT ROTATION IS DESCRIBED 
RotationAngle(5): ANGLE OF ROTATION 
RotationSpeed(5): SPEED OF ROTATION 
RotatlonAcceleration: ACCELERATION OF ROTATION 



FIG. 2B 
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FIG. 3A 



SmellObjectFlagd bit): WHETHER ODOR DATA ARE DESCRIBED 
Length(32bits): PACKET SIZE OF ODOR DATA 
Type{4bits): KIND OF ODOR 
Level(5bits): INTENSITY OF ODOR 



FIG. 3B 
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FIG. 4A 



TasteObjectFlagdbit): WHETHER TASTE DATA ARE DESCRIBED 
Length(32bits): PACKET SIZE OF TASTE DATA 
Typed Obits): KIND OF TASTE 
Level(5blts): INTENSITY OF TASTE 



FIG. 4B 
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FIG. 5 
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